Systems can use three-dimensional (“3D”) models to represent objects. For instance, an augmented reality (“AR”) system, virtual reality (“VR”) system, or web browser can use 3D models to represent objects in the corresponding environment. The models might be outside a field of view and affect presentation of objects inside the field of view, e.g., a ball that is bouncing on a chair.